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Multicolor urn models with reducible 
replacement matrices 
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Consider the multicolored urn model where, after every draw, balls of the different colors are 
added to the urn in a proportion determined by a given stochastic replacement matrix. We 
consider some special replacement matrices which are not irreducible. For three- and four-color 
urns, we derive the asymptotic behavior of linear combinations of the number of balls. In par- 
ticular, we show that certain linear combinations of the balls of different colors have limiting 
distributions which are variance mixtures of normal distributions. We also obtain almost sure 
limits in certain cases in contrast to the corresponding irreducible cases, where only weak limits 
are known. 

Keywords: martingale; reducible stochastic replacement matrix; urn model; variance mixture of 
normal 

1. Introduction 

Consider an urn model with balls of K colors. The row vector Co will denote the number 
of balls of each color we start with. (By abuse of terminology, we shall allow the number 
of balls to be any non-negative real number.) The vector Co will be taken to be a 
probability vector, that is, each coordinate is non-negative and the coordinates add up 
to 1. Suppose i? = ((''ij)) is a A' X K non-random stochastic (i.e., each row sum is one) 
replacement matrix. The results of this paper extend to non-random replacement matrices 
with constant (not necessarily one) row sums by an obvious rescaling. Let d be the row 
vector giving the number of balls of each color after the nth trial. At the nth trial, a ball 
is drawn at random, and so a ball of ith color appears with probability C.„_i/n. If a 
ball of ith color appears, then the number of balls of jih color is increased by rij. If R 
equals the identity matrix, then it is well known (see, e.g., [3]) that Cn/{n+ 1) converges 
almost surely to a Dirichlet random vector with parameters given by the starting vector 
Co. 

Let 1 or stand respectively for the column vector of relevant dimension with all 
coordinates 1 or 0. For any vector ^, will be the vector whose coordinates are the 
square of those of 
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In Section 2, we consider two color models {K = 2). If the replacement matrix R is 
not the identity matrix, then it has two right eigenvectors, 1 and ^ corresponding to the 
principal eigenvalue 1 and the non-principal eigenvalue A, respectively, with |A| < 1. If i? 
is irreducible, the asymptotic properties of C„l and Cn^ are well known in the literature, 
see Proposition 2.1. 

When the replacement matrix R is reducible but not the identity matrix, then, after 
possibly interchanging the names of the colors, R is an upper triangular matrix 



for < s < 1. Here the non-principal eigenvalue is s with the corresponding eigenvector 
^ = (1,0)'. The asymptotic behavior of the linear combinations is given in Proposition 2.2. 
In this case, C„^/n^ = Wn/n^ converges almost surely for all values of s in contrast to the 
irreducible case. See also Theorems 1.3(v), 1.7, 1.8 and 8.8 of [7], where the distribution 
of the limiting random variable was identified using methods from the branching process. 

In the multicolor case, when R is irreducible, the weak/strong laws corresponding 
to different linear combinations arc completely known, see [1, 6]. Gouet [4] considered 
(reducible) replacement matrices that are block diagonal, with all but the last block irre- 
ducible. The last block was taken to be block upper triangular, which cannot be converted 
into a block diagonal one, and each diagonal subblock of the last block was assumed to be 
a multiple of some irreducible stochastic matrix. He showed (cf. Theorem 3.1 of [4]) that 
the proportions of colors converge almost surely to a constant vector where the non-zero 
coordinates correspond to all but the last diagonal block and the last diagonal subblock 
of the last diagonal block. We call the corresponding colors dominant. To avoid trivial 
situations, we shall always assume positive contribution to at least one non-dominant 
color in the initial vector Cq. 

We shall consider three- and four-color urn models with block upper triangular re- 
placement matrices that are not block diagonal. The diagonal blocks will be taken to be 
irreducible and wc shall extend the result obtained in [4] by obtaining the limiting results 
for linear combinations corresponding to a complete set of linearly independent vectors. 

Specifically, in Section 3, we consider three colors - white, black and green - and the 
3x3 replacement matrix 



where < s < 1, and Q is a 2 x 2 irreducible aperiodic stochastic matrix with stationary 
distribution ttq . Here green alone is the dominant color and we assume that Wo + i?o > 0. 
We show in Theorem 3.1(iv) that (W„,B„)/n^ -nqV, where P{V > 0) = 1 and 1/ is 
non-degenerate. If ^ is the eigenvector corresponding to the non-principal eigenvalue A of 
Q, weak/strong laws for [Wn, Bn)£, are also provided in Theorem 3.1. If A < 1/2, then the 
weak limit is a variance mixture of normal, in contrast to the irreducible model, where 
the weak limit is normal. 




(1) 




(2) 
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In Section 4, we consider another type of reducible replacement matrix with two dom- 
inant colors: 



where P is a 2 x 2 irreducible stochastic matrix, p is a row probability vector and 
< s < 1. If the eigenvalues of P are A and 1, then s, X and 1 are eigenvalues of R. Clearly 
(1,0, 0)' is the eigenvector corresponding to s and the behavior of the corresponding linear 
combination, Wn, follows directly from Proposition 2.2. 

Now consider the eigenvalue A. If R is diagonalizablc, then the weak/strong law of 
the linear combination given by the eigenvector corresponding to A is summarized in 
Theorem 4.1. If R is not diagonalizable, then one of the eigenvalues is repeated, namely 
A = s, and the repeated eigenvalue has eigenspace of dimension 1, spanned by (1,0,0)'. 
Consider the Jordan decomposition of R, RT = T J, where T is non-singular and 



The first and the third columns of T can be chosen as (1,0, 0)' and 1 respectively. For the 
linear combination corresponding to the middle column, we get weak/strong law. The 
convergence is in the almost sure sense, whenever A > 1/2, unlike the irreducible and 
the diagonalizable reducible cases. For A = 1/2, in the irreducible and the diagonalizable 
reducible cases, we have weak convergence only. Also, the scaling for the irreducible 
case is \/ n \og^ n and for the diagonalizable reducible case is \/nlogn, unlike the non- 
diagonalizable reducible case, where the scaling is \/ nlog^ n. 

Other interesting reducible three-color urn models have been considered in the litera- 
ture. For example, [2] and [9] consider three-color urn models with triangular replacement 
matrices. Our emphasis is on replacement matrices with block triangular structure, given 
by (2) and (3). Note that Q in (2) is assumed to be a stochastic matrix. However, our 
techniques do not have a direct extension to the case where Q does not have a constant 
row sum. In another related work, Pouyanne [8] allows eigenvalues of the replacement 
matrix to be complex and obtains interesting results for appropriate linear combinations. 
For example, in his Theorems 3.5 and 3.6, rates are given for the linear combinations 
corresponding to the eigenvalue with the second largest real part, where it is bigger than 
1/2. In our setup, all the eigenvalues are real and we obtain the rates for all possible 
linear combinations. 

The results for three-color urn models are extended to four-color (white, black, green 
and yellow) urns with the reducible replacement matrix given by 




(3) 




(4) 




(5) 



where each component is a 2 x 2 matrix and furthermore P and Q arc irreducible stochas- 
tic matrices, < s < 1. The results are summarized in Propositions 4.2-4.5. An interesting 
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phenomenon is observed in Proposition 4.5, where the replacement matrix is not diago- 
nalizable and the repeated eigenvalue is zero. Unlike the behavior of the corresponding 
linear combination in other cases, where it remains a constant, we get a weak limit of 
variance mixture of normal distribution in this case. 

Before proceeding with the details, note that the proofs arc based on studying the 
behavior of appropriate martingales with the filtration Tn being the natural filtration of 
the sequence {C„}. 

2. Two-color urn models 

Define 

n„(A)=n(i+-A_y (6) 

i=o V / 
Recall that Eulcr's formula for gamma function gives 

nn(A) ~ 7i'^/r(A + 1), A not a negative integer. (7) 

This will be used at several places later. 

We first mention the asymptotic behavior in two-color irreducible urn models. The 
following results are well known. See, for example, [1, 6]. 



Proposition 2.1. In a two-color urn model with irreducible replacement matrix R, 



a.s. 



^Ti?, (8) 



n+1 

where tt/j is the stationary distribution of R. Further, we have: 

(i) //A<l/2, thenCn^/y^^N{0, j^TZRe). 

(ii) //A = l/2, then C„4/V"logn =^ iV(0, A^Tr^i^). 

(iii) If X > 1/2, then Cn^/Iln{X) is an L^-bounded martingale and converges almost 
surely, as well as in , to a non-degenerate random variable. 



Remark 2.1. Since Wn + -B„ = n + 1, we have from (8), 

(W^„,B„)/(W^„ + B„)'^-7rfl. (9) 



Remark 2.2. From (8), we see that C„^/(n + 1) ivi^^,. However tt/?^ = t^rR^ = 
Xtth^, and since A 7^ 1, we have tth^ = 0. This explains the appropriate scaling up of 
C„^/(n+ 1) to obtain the weak laws for the proportions above. 

Remark 2.3. When A > 1/2, using (7), Cn^/n^ converges almost surely, as well as in 
L^, to a non-degenerate random variable. Thus, the scalings in Proposition 2.1(i) and 
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(iii) are different. Also, in (iii), the distribution of tlic limit random variable depends on 
the starting value (Wo,Bq) unlike in (i) and (ii). Furthermore, as in Remark 2.1, we can 
conclude that, when A > 1/2, {Wn, B„)^/ {Wn + Bn)^ converges almost surely, as well as 
m L'^, to a non-degenerate random variable. 

Remark 2.4- If A = 0, both rows of R equal ttji. Since tt/j^ = and clearly C„ = 
Co + nTTfi, we have = Co^ for all n. 

Next, we consider the almost sure limit behavior of the two-color urn model with an 
upper triangular reducible replacement matrix given by (1). 

Proposition 2.2. In a two-color urn model with an upper triangular replacement matrix 
given by (1), we have: 



(iii) C„4/n„(s) = W^„/n„(s) is an L'^-bounded martingale, where n„(s) is given 
by (6). Further, Wn/n^ converges to a non- degenerate, positive random variable 
almost surely, as well as in L? . 

Proof. Statement (i) is trivial. Statement (ii) is same as that of (8) in Proposition 2.1 
and a proof can be obtained from Proposition 4.3 of [4]. 
For (iii), observe that the number of white balls evolves as 



where Xn is the indicator of a white ball in nth trial. Define the martingale sequence 
Vn = W^n/n„(s), n > 1. We shall show that {Ki} is an L^-boundcd martingale and hence 
converges almost surely, as well as in I? . Also, the variance of Vn increases to that of the 
limit and hence the limit is non-degenerate. The proposition then follows from (7), the 
distribution of V and the fact that it is almost surely positive, all of which have been 
established using branching process techniques in Theorem 1.3(v) of [7]. 
Clearly, we have 



and further, using Vn+\ =Vn + {Vn+i — Vn) and the martingale property, there exists N 
(non-random), such that for all n> N, 



(i) Cnl/{n + l)^l. 

(ii) C„/(n+l)"^-(0,l). 



Wn+l =Wn+ SXn+1, 




n2 



-n4-l 



(s)[n+l (n + l)2 




(n + l)n„(s) 



<y„2 + r(s + i) 



14- y2 



1 4- 



r(s + i) 1 r(s + i) 

(n-hl)^+ij (n-f 1)^+1 ■ 



(n -I- 1)^+1 
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The last inequality holds for n> N and follows from the fact that Vn < {1 + V^)/2 
and (7). Taking further expectation and adding 1 to both sides, we have, for n> N , 



Iterating, we get for n> N, 



r(s + i) 



{71 + 1)"+^ 



E[Vn+i] + 1 < iE[V^] + 1) n 
and since s > 0, we further have for all n> N , 



(oo 
r(5 + i)^, 

which shows {Vn} is L^-bounded as required. 



{E[V?A + !)• 



r(s + i) 



□ 



3. One dominant color, K = ^ 

Now we are ready to consider the three-color urn model with only one dominant color, 
say green. We shall denote the row subvector corresponding to the non-dominant colors 
[WmBn] as Sn- We collect the results in the following theorem. 

Theorem 3.1. Consider a three-color urn model with a reducible replacement matrix 
R given by (2). Suppose the non-principal eigenvalue of Q is A and the corresponding 
eigenvector is ^. Then the following hold: 

(i) C„l/(n + l) = l. 

(ii) C„/(n+l)"^- (0,0,1). 

(iii) SI„l/(n-|- 1)* converges almost surely, as well as in L'^ , to a non- degenerate pos- 
itive random variable U. 

(iv) Sn/in + iy'^-TZQU. 

(v) //A<l/2, then Sn^/n'/^ ^ NiO,j0^U7ZQe). 

(vi) //A = 1/2, then Sn^/Vn^Tog^^ N{0, s^X^U-Kqi^). 

(vii) If X> 1/2, then S'„^/n„(sA) is an L^-bounded martingale and almost surely, as 
well as in , Sn^/n"'^ — ^ V , where V is a non-degenerate random variable. 

The random variable U in (iv), (v) and (vi) is the same limiting random variable obtained 
in (iii) . The distributions of U and V depend on the initial value So ■ 

Remark 3. 1 . Note that the eigenvalues of R are 1 , s and sA with corresponding eigen- 
vectors 1, (1,1,0)' and (^',0)', respectively, yielding the linear combinations C„l, S^il 
and Sn$,- 
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Proof of Theorem 3.1. Statement (i) is immediate. Statement (ii) follows from The- 
orem 3.1 and Proposition 4.3 of [4]. 

Note that 5'„1 = Wn + Bn- From the structure of R, the pair {Wn + G„) yields a 
two-color model with reducible replacement matrix 



1 - s 
1 



Statement (iii) then follows from Proposition 2.2. The distribution of U has been iden- 
tified in Theorem 1.3(v) of [7]. 

Consider the successive times when either a white or a black ball is observed. Due to 
the assumed special structure of the matrix R, it is only at these times that more white 
or black balls arc added and the total number added is the constant s. Thus St^/ St^. 1 are 
the proportions from the evolution of a two-color urn model governed by the irreducible 
replacement matrix Q. Hence by the two-color urn result (9), it converges almost surely 
to tvq. Note that at all other n, < n < Tk+i, the vector Sn = Sr^. and hence the ratio 
is unchanged. Moreover, from the statement (iii), we have Sn'^/n'' ^-—^ V . Now combining 
all of the above, the proof of the statement (iv) is complete. 

For (vii), let Xn be the row vector, which takes values Xn ~ (liO)i (0; 1) or (0,0) 
accordingly as the white, black or green balls are observed in nth trial and consider the 
martingale r„ = S'„^/n„(sA). Then the martingale difference is 



T, 



n+l 



sA 



n„+i(sA) 



and hence 

E[T^^,]=E[n] 



sX 



n„+i(sA) 



1 - 



E 



(n+ 1)2(1 + sA/(n + l))2 



sA 



1 



n„+i(sA); (n + l)i 



-E 



The first term is bounded by E[T^]. From statement (iii), S'„l/(n + 1)* is L^-bounded 
and hence i^-bounded. So S',i$^/ (n+ 1)** is also i^-bounded. Thus, using (7), the second 
term is bounded by a constant multiple of n~'^+''(2^-i)) ^ which is summable as A > 1/2. 
Thus {T„} is an L^.^^Q^j^^jgf;! martingale and hence converges almost surely as well as in 
L\ 

For (v) and (vi), we start with the case A < 1/2. Call Xn = Sn$,/n^^^- Wc have the 
evolution equation for Sn^ given by 



Sn+l^ = Sn$, + SXn+lQ^ = Sn$, + XsXn+1^- 



(10) 



We now use the decomposition of Xn+i into a conditional expectation and a martingale 
difference 



Xn+l — E{Xn+l\^n) + {Xn+1 — E{Xn+l\J-n)} ■ 
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Using (10) and the fact that (1 + l/n)-^/^ = (i _ s/2n) + Oil/n^) we then get 

i?(X„+i|^„) = ^(1 + + . 

n"/^ (n + 1)*/^ n + 1 

/ 1 \ \ / 1 \ -«/2 



= X„ 1- — + - +AsX„ 1 , 

\ 2n J J \ n J n + 1 

= X„ (^1 - ^^^^1^) + ^«0(n-2). (11) 
On the other hand, the martingale difference is reaUy 

M„+i :=X„+i - £;(X„+i|^„) = ^_^(^x„^, - -^y, (12) 



X„+i = X„ ( 1 - '^^^^ ) + X„0(n-2) + M„+i. (13) 



so that 



Iterating the equation above, we get 

x„,..x.n(i-£M).tx,o,r^, fi {^-^ 

Since A < 1/2, we have n„(-s(l/2 - A)) - ri-'*(i/2-^Vr(l - s(l/2 - A)) ^ 0, and hence 
the first term above converges to for every sample point. The continued product in 
the second term is bounded by 1. Since the coordinates of S'„/(n + 1) are bounded by 
1, we have that is bounded for every sample point. Thus the sum of the 

elements of the second term above is bounded by a multiple of which is 

finite; and individually each element tends to zero since each infinite product diverges to 
zero. Hence the second term of (14) tends to zero for every sample point. 
Now we turn to the third term of (14), 

Z^^,=±M,^, n (15) 

We verify the conditional Lyapunov condition and compute the conditional variance as 
n ^ oo. The conditional Lyapunov condition demands that for some A; > 2, 

±E{\M,^,m) n (i-^^^^^)'^^-o. 
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Since each coordinate of Xn+i ^^^^ Sn/{n + l) is bounded by 1, the martingale difference 
defined in (12) is bounded by a constant multiple of (n + l)"**/^. Thus the above sum is 
bounded by a constant multiple of 

j=l i=3 + l ^ 

which tends to zero by the bounded convergence theorem, provided we choose k > 2/s. 

Now we compute the conditional variance. An exact computation and the statement 
(iv) yields, with probability 1, 



(Xs) 



n+1 \n+l 



n. + l 



Then, writing nL,+i(l - ^^i^^) = n„(-s(l/2 - A))/nj(-s(l/2 - A)) and using (7), 
the sum of the conditional variances satisfies, on a set of probability 1, 



js(l-2A) Zl^ jl-s{l-2X) ' 

j=i i=j+i ^ ^ ' ■ i=i-' 

which converges almost surely to (As)^J7ttq^^/s(1 — 2A). Thus, by martingale central 
limit theorem (see Corollary 3.1 of [5]), the limiting distribution of Zn+i, and hence 
Xn+i is the required variance mixture of normal. 

Since the analysis for the statement (vi) is similar, we omit the details and provide 
only a brief sketch of the arguments. We start with X„ ~ Sn^/^/n^logrl. The following is 
the relevant martingale decomposition now. To express the decomposition, the following 
straightforward approximations are used: 

(l + l/n)-/2 = l-^ + 0(n-^) and ^^^n logn 



2n log(n+l) logn+ l/7i + 0(n-2) 



together give 

s/2 



log" A « , ^r..-2^\ f, 1 , r^/^ 1 



1 - — + Oin-') 1 - — + O 



n + 1 J y log(n + l) \ 2n J\ 2n\ogn yn^logn 

= 1- — + 0(n-2). 

2n 2n\ogn ^ ' 

Using A = 1/2 carefully, the conditional expectation becomes 

-B(X„+i|.F„) =^„[1- (2nlogn)-i] +X„0(n-2) 

and, for the martingale difference, we get. 
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These together give us the recursion on cis 

X„+i = X„[l - (2nlogn)-i] + X^O{n-^) + A/„+i, 

a decomposition similar to (13). The rest of the proof foUows as before with appropriate 
changes. □ 

Remark 3.2. Theorem 3.1 gives the scahng for aU the hncar combinations except when 
A = 0, in which case (v) apphcs and wc obtain Syi^/n^/^ 0. However, as discussed in 
Remark 2.4, Q has both rows the same as ttq, which satisfies ttq^ = 0. Since Sn changes 
only when a white or black ball appears, we have Sn = 5o + {Wn + Bn)'KQ and hence 
Sn£, — Sq$, for all n. 



4. Two dominant colors, = 3,4 

Wc now consider the three-color case with two dominant colors. The replacement matrix 
R, given by (3), is 

(s [l-s)p\ 

where p is a probability vector and P is a 2 x 2 irreducible stochastic matrix. Thus 
1 is always an eigenvalue of P with the corresponding eigenvector 1. We shall denote 
the other eigenvalue of P as A with corresponding eigenvector ^. Then s and 1 are two 
eigenvalues of R with corresponding eigenvectors (1,0,0)' and 1, respectively. Observe 
that C„(1,0,0)' = Wn- The results for this linear combination follow from two-color urn 
model results, and we summarize them below. We shall denote the stationary distribution 
of P by TV p. 

Proposition 4.1. Consider a three-color urn model with two dominant colors and the 
replacement matrix given by (3). Then: 

(i) Cnl/{n + l)^\. 

(ii) C„/n'-4-(0,7rp). 

(iii) Wn /n'^ ^ V almost surely, as well as in L? . 

In (iii), «/ we start with the initial vector Cq = {Wq, Bo,Go) , then V has the same 
distribution as the limit random variable in Theorem 3.1(vii) with the initial vector 
[Wo, Bo + Go). 

Proof. Statement (i) is trivial. The proof of (ii) is given in Theorem 3.1 or Proposition 4.3 
of [4]. For the remaining part, consider the two-color urn model (M^„, P„ -I- G„) obtained 
by collapsing the last two colors. This will have the replacement matrix as in (1) and the 
results will follow from Proposition 2.2. □ 
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However, the one remaining linear combination is more subtle. The choice of the linear 
combination depends on whether R is similar to a diagonal matrix or, equivalcntly, has 
a complete set of eigenvectors. Suppose A ^ s. Then R is diagonalizable and V2 — (c,^')' 
is an eigenvector of R corresponding to A, with c = (1 — s)p^/ (A — s). If A = s, then R 
is diagonalizable if and only if ~ 0. In that case, (0,^')' is another eigenvector of R 
corresponding to s independent of (1,0,0)'. Also note that, in that case, p is orthogonal 
to ^ and since p is a probability vector we have p = TVp. In the diagonalizable case with 
A = s, we denote this remaining vector (0,^')' by V2 and consider the corresponding 
linear combination. The following theorem summarizes the results. 

Theorem 4.1. Consider a three-color urn model with replacement matrix R given by (5), 
where R is diagonalizable. Then the following weak/strong laws hold: 

(i) //A<l/2, thenCnV2/V^^N{0,j^TVp^^). 

(ii) //A = 1/2, thenCnV2/Vrnogn^ N{0,X'^7Vp$,^). 

(iii) If X > 1/2, then C„f2/n„(A) is an L'^-bounded martingale and CnV2/n^ con- 
verges almost surely to a non-degenerate random variable. 

Proof. The proofs of (i) and (ii) are similar to those of (v) and (vi) of Theorem 3.1, so 
we omit them. 

Define %„ as the row vector that takes values (1,0, 0) , (0, 1, 0) and (0, 0, 1) accordingly 
as white, black or green balls appear in nth trial. Also define Z„ = C„t>2/n„(A). It is 
simple to check that {^„} is a martingale. Note that, 

y y ^ f ^ 

Z„+i - Z„ - — — - Xn+l TT P2, 

n,i+i(A) V ^ n+lj 



which gives us 



E[[Zn+l — ZnY\Tn] — 



A2 



n^.+i(A) 



CnVl ( CnV2 



Also, Cn/{n + 1) is bounded by 1 for each coordinate. Hence, the conditional expectation 
above is bounded by a constant multiple of n~^^. So we get E\Z\^^ = X]r=i{-^[(-^«+i ~ 
Zi)^} is bounded by a constant multiple of X^-T*"^"^' which is finite, as A > 1/2. Thus, 
{Zn} is i^-boundcd. The rest of the statement (iii) follows from (7). □ 



If R is not diagonalizable, then a complete set of eigenvectors is not available and 
one of the eigenvalues must be repeated, which gives s = A and p 7^ TTp. So we consider 
the Jordan decomposition RT = TJ, where J is given by (4). We can choose the first 
and third columns of T as ti = (1,0,0)' and = 1. Also the subvector of the lower two 
coordinates of t2 is an eigenvector of P corresponding to s. We shall denote it by ^ as 
well. The behavior of C„i2 is substantially different from the irreducible case given in 
Theorem 3.15 of [6] or the diagonalizable case in Theorem 4.1 above. 
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Theorem 4.2. Consider a three-color urn model with replacement matrix R given by (5), 
where R is not diagonalizable. Then, we have: 

(i) Ifs<l/2, then Cnt2/V^^ N{0,j^^np^^). 

(ii) If s > 1/2, then Cnt2/n^^ogn converges to V almost surely, as well as in , 
where V is the almost sure limit random variable obtained in Proposition 4.1(iii). 

Proof. We first consider the case when s < 1/2. Call X„ = Cntil\fn. Define the row 
vector Xn in t^ic proof of Theorem 4.1. We shall split X„+i into conditional expectation 
and martingale difference parts as in the proof of Theorem 3.1(v). From the Jordan 
decomposition of R and the form (4) of J, the evolution equation for C„ is given by 

Cri+lt2 = Cnti + SXn+\^1 + Xri+l*l- 

Hence the conditional expectation becomes 

|^„) = f 1 + ^\ + 

\/n+l\ n+1 J (n+iy/^ 

since Cnii = W„. Using the notation st = t\ + st2, the martingale difference term be- 
comes 

Mn+l ■■= X.a+1 - E{Xn+i\J^n) = . , ^ ( Xn+1 ^ 

V«+l\ n+1 
Putting this together, we get a recursion on X„ as 



1/2 , ^ , W„ 

(n+ 1)3/2 



X„+i = X„ 1 - ^ + X„0(n-2) + " + A/„+i, 



and iterating we get, 

x.+i^x,fl(i-'-I^)+±xMr^) n fi-^ 



j=l i=j+l 



J=l ^-^ ^ i=J+l ^ / i=l ^ ^ 

which is similar to the decomposition (14), except for the additional third term. Further 
analysis is similar to that done for Theorem 3.1(v), except for the contribution of the 
third term, which we now show to be negligible with probability 1. By Proposition 4.1(iii), 
writing n" .+i(l- i^) = n„(-(l/2-s))/n, (-(l/2-s)) and using (7), the third term 
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is of the order of 



V 



Vlogn 



1 " 

„l/2-s $Z j3/2-s j-{l/2-s) ' „l/2-s ' " 
3 = 1 ■' 

almost surely, since s < 1/2. 

Using Proposition 4.1(ii), the structure of the vectors t\ and t-j and the fact ■np$, = 0, 
the conditional variance term is 



n + 1 



n+1 



Cyit 

11+1 



n+1 



which gives the required variance for the limiting normal distribution. 

Now we consider the other situation, where s > 1/2. Using the form (4) of J in the 
Jordan decomposition of i?, we again have Rt2 = f i + st2 = st. Thus 



which implies 



E[Cn+lt2\^n] — Cnt2 1 



n + 1 



n + 1 



This gives us the martingale 



^ _ Cnt2 \r-^ 1 Cjt 



Cjti 



(16) 



The martingale difference is then given by Xn+i — Xn = s(Xn+i ^ )*/nn+i(s), which 
yields 



£;[(x 



n+1 



Cnt 



n + 1 



Cnt 

n + 1 



(17) 



and using the fact that each coordinate of Cn/{n + 1) is bounded by 1 and Euler's 
formula for gamma function, the conditional second moment above is bounded by a 
constant multiple of n^'^^ . Taking expectation and adding, we get £'[X,'^_(_j^] is bounded 
by a constant multiple of X^o This implies, {Xn} is L^-bounded if s > 1/2 and, 
{Xn/ ^/\ogn} is L^-bounded if s = 1/2. Thus, for s > 1/2, X„/logrt almost surely, 
as well as in L^. For s = 1/2, Xn/ \ogn ^ in L^. 

We now show the convergence is almost sure also, when s ~ 1/2. For this, consider the 
random variables Z„ = X„/logn and get 



n+1 



Zn 



X. 



n+1 



Xn 



X,, 



log(n+l) logn 



logn 



log(n+l) 



(18) 
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Since [1 — j^^°^^"-j^^ ]/^/logn ~ \/n\o^^^ n and X„/-y/logn is L^-bounded (and hence in- 
bound 
hence 



log(ri+l)J 

bounded), we have £'Efe=2 ^'/[''\ // ~ log"^-^!) bounded uniformly over n and 



k=2 vloP Vlop 



log(fc + 1) 



converges absolutely almost surely. On the other hand, the first term of (18) is a 
martingale difference and, using (17) for s = 1/2, the conditional variance E[{Xn+i — 
-'^n)^/log^(n + l)|.?>i} is bounded by a constant multiple of [(n + 1) log^(n + 1)]~^, 
which is summable. Hence, the martingale {^^^^(X^+i — Xfc)/log(fc + 1)} is L^-bounded 
and thus converges almost surely. Combining the two observations above we get that 
Zn = Xn/logn converges almost surely. 

Thus X„/logn converges to almost surely, and in L^, for all s > 1/2. Hence, 
from (16), we have 

Xn Cnt2 1 1 Cjti 



— S^— (19) 

gn^j + in,+i(5) ^ ^ 



log?i lognH„(s) log" .^gj + in,+i(s) 

converges to almost surely, as well as in L^. But using (7) and Proposition 4.1(iii), we 
know that C,iti/n„(.s) ^ T{s + \)Wn/n^ r(s + 1)^ almost surely, as well as in L^. 
Hence the second term in (19) converges to r(s + 1)V almost surely, as well as in L^. 
Thus, 

C„t-2 1 Cnt2 



n*logri r(s + 1) n„(s) logn 
converges to V almost surely, as well as in L^. □ 

Remark 4-1- As in the case of one dominant color, we have the correct scaling for all 
the linear combinations except when A = < s. (This situation arises only in the case of 
diagonalizable replacement matrix.) But V2 being an eigenvector of R corresponding to 
A = 0, we have Rv2 = 0. Thus C„t>2 = Cof 2 for all n. 

The three-color urn model with two dominant colors can be easily extended to certain 
four-color models. We consider the reducible replacement matrix given in (5), 



R- 



sQ E 
P 



where P and Q are 2x2 irreducible stochastic matrices, < s < 1. The eigenvalues of 
Q are A and 1, with |A| < 1. The eigenvalues of P are (3 and 1, with |/?| < 1. Then sA, 
s, (3 and 1 are all eigenvalues of R. If ^ is an eigenvector of Q corresponding to A, then 
vi = (l',0')', V2 = (^ ,0')' and v^ — l are eigenvectors of R corresponding to s, sA and 
1 respectively. 
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If R is diagonalizable, then there is another eigenvector ^3 corresponding to /?. If R 
is not diagonalizable, then one of its eigenvalues must repeat, namely (3 must equal s or 
sA and we denote the other by a. In this case, we consider the Jordan decomposition 
RT = TJ, where T is nonsingular. The fourth column ti of T can be chosen as V4. The 
first two columns ti and t2 of T can be chosen as the eigenvectors of R corresponding 
to a and /?. However, the third column of T will not be an eigenvector of R, yet 
the two-dimensional vector v formed by the lower half of ts will be an eigenvector of P 
corresponding to (3. We shall only study Cnt^ separately in the non-diagonalizable case. 

The following three Propositions are suitable extensions of the three-color results of 
this section. The proofs are suitable modifications as well. 

Proposition 4.2. Consider a four-color urn model with the replacement matrix given 
by (5). Then: 

(i) C„l/(n + !) = !. 

(ii) C„/n"^ (0,0,7rp). 

(iii) (W„,B„)K"-^-7rQ?7. 

(iv) CnVi/n'^ U almost surely, as well as in . 

(v) If X< 1/2, then C„V2/n'^^ => N{0, -^^Utvq^). 

(vi) If X = 1/2, then CnV2/Vn^\og^=^ N{0, s^X^U-Kqi^). 

(vii) If X> 1/2, then CnV2/n^^ — *■ V almost surely, as well as in . 

If we start with the initial vector (Wq, Bq,Go,Yo) , then U and V have the same distri- 
bution as the limit random variable in Theorem 3.1 (iii) and the positive random variable 
in Theorem 3.1(vii), respectively, starting with initial vector (Wq, Bq,Go + Yo). 

Next we consider the linear combination CnVs in the diagonalizable case. 

Proposition 4.3. In the four-color urn model with replacement matrix R given by (5), 
assume that all the eigenvalues of R are distinct. Then the following weak/ strong laws 
hold for CnV^: 

(i) If P< 1/2, thenCnV3/y^=>NiO,j^7Vpi^''). 

(ii) If (3^1/2, then CnVs/Vnlogn^ N{0, (3^TVpu^). 

(iii) If (3 > 1/2, then C„t>3/n„(/3) is an L^-bounded martingale and CnVz/rfi con- 
verges almost surely to a non- degenerate random variable. 

Finally, we consider the case when the replacement matrix R is not diagonalizable. As 
in the three-color urn model with a non-diagonalizable replacement matrix, the evolution 
of the linear combination Cnt^ depends on the eigenvector of R corresponding to the 
eigenvalue p. When (3 < 1/2, the effect of the contribution of the linear combination of 
this eigenvector is negligible. However, for (3 > 1/2, this provides the main contribution 
and the almost sure limit random variable depends on whether f3 equals s or sX. To 
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denote the limit random variable in a unified way, we define the random variable 

w = [^; "^1:'"^ = ^ (20) 

V/, when p = sX, ^ ' 

where U and V are the random variables defined in Proposition 4.2. Suppose f3 > 1/2. If 
P = s, then t2=vi and, by Proposition 4.2(iv), Cnt2/in+l)f' = CnVi/{n + lY ^ U = W 
almost surely, as well as in L^. If /3 = sA, then t2 = V2- Also s < 1 implies A > 1/2. Hence 
by Proposition 4.2(vii), Cnt2/{n + l)'^ = CnV2/{n + 1)'''^ — W almost surely, as well 
as in L^. So, for non-diagonalizable R and f3 > 1/2, we conclude C„t2/(n + 1)^ — *■ W 
almost surely, as well as in L^. 

Proposition 4.4. Consider the four-color urn model with replacement matrix R given 
by (5), where R is not diagonalizable. Then, we have: 

(i) If P< 1/2, thenC„h/^^NiO,j^7TpU^). 

(ii) If f3 > 1/2, then Cnta/n^logn converges to W almost surely, as well as in L? , 
where W is as defined in (20). 

Remark 4-2. As in two- and three-color urn models, we have correct scalings for all 
linear combinations except when A or /3 becomes zero. If A = 0, Proposition 4.2(v) gives 

CnV2/n'^^'^ 0. However, considering the three-color urn model (W^„,i3„,G„ + Yn), we 
get, from Remark 3.2, CnV2 = CoV2. 

In the case of the diagonalizable replacement matrix, if f3 — 0, we have a similar situ- 
ation for the linear combination C„V3 in Proposition 4.3(i). However, as in Remark 4.1, 
^3 being an eigenvector of R corresponding to /3 = 0, we have Rv^ = and C„i?3 = CqV^. 

The situation becomes more interesting when /3 = and the replacement matrix is 
not diagonalizable. Thus, we necessarily have P = s or P = As, but (3 being zero and 
s being positive, only the second alternative is possible and further /3 = A = 0. In this 

case. Proposition 4.4(i) gives Cnts/^/n—f 0. The correct rate is given in the following 
proposition. 

Proposition 4.5. Consider the four-color urn model with the replacement matrix R 
given by (5), which is not diagonalizable. Further assume the repeated eigenvalue of R to 
be zero. Then 

Cnt3/n'/^^NiO,TTQeU/s), 

where U is the limit random variable corresponding to [Wn + Bn)/n'^ obtained in Propo- 
sition 4.2(iv). 

Proof. Let Xn be the row vector as in the proof of Theorem 3.1(vii). Using RT = TJ 
and the fact /3 = 0, we get Rt^ = t2 + /3*3 = *2- Hence using t2 = (^', O')', the evolution 
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equation for Cnt^ is given by 

The rest of the proof is similar to that of Theorem 4.2(i) and is omitted. □ 
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